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Abstract — In this paper the multicasting of independent paral- 
lel Gaussian sources over a binary erasure broadcasted channel 
is considered. Multiresolution embedded quantizer and layered 
joint source-channel coding schemes are used in order to serve 
simultaneously several users at different channel capacities. 
The convex nature of the rate-distortion function, computed by 
means of reverse water-filling, allows us to solve relevant convex 
optimization problems corresponding to different performance 
criteria. Then, layered joint source-channel codes are constructed 
based on the concatenation of embedded scalar quantizers with 
, binary rateless encoders. 

I. Introduction 

Multimedia streaming over heterogeneous digital networks 
is one of the fastest growing applications both in terms of 
traffic demands and in terms of market potential. A conven- 
tional approach consists of establishing individual streaming 

■ sessions from the server to each users. This may be very 
inefficient when many users wish to receive the same content 
(e.g., in mobile television or video on demand applications). A 
definite advantage of analog broadcasting systems is sending 
simultaneously the same signal to a potentially unlimited num- 
ber of receivers, with possibly different reconstruction quality 
that depends on the channel conditions. Analog transmission 
finds its theoretical justification in the fact that a Gaussian 

■ source under the quadratic distortion measure is "matched" to 
an Additive White Gaussian Noise (AWGN) channel [1]. Un- 
fortunately, this lucky "matching" condition does not generally 
apply to a heterogeneous digital network (e.g., the Internet). 
Such a network, which may be formed by wireline backbone 
infrastructure combined perhaps with wireless component to 
the end users, is characterized by a large variability of the 
"capacity" from the media server to the individual users. 
The protocol stack is basically fixed. Therefore multimedia 
streaming applications are typically designed as "overlay" 
systems, that is, they act at the application layer by using 
a fixed transport mechanism already built-in to the network. 
In this paper, following [2], [3], [4], we model the transport 
mechanism as a binary erasure broadcast channel (BEBC). 
This model captures the essential behavior of a network in 
which the server sends a stream of packets without end-to-end 
flow control (e.g., using User Datagram Protocol, UDP [5]) 
and these may be lost because of errors or buffer overflows in 
the network. The receivers have perfect knowledge of packet 



losses since packets contain sequence numbers, and therefore 
can treat missing packets as erasures. 

Motivated by the above scenario, we focus on efficient 
Joint Source-Channel Coding (JSCC) for the transmission of 
a source over the BEBC. We refer to this approach as JSCC 
"at the application layer" since we disregard the underlying 
physical channels, in contrast to most JSCC approaches pre- 
sented in the literature (e.g., [6] and references therein). In this 
paper we extend our approach for JSCC of an independent and 
identically distributed (i.i.d.) Gaussian source over the BEBC 
proposed in [3] to the case of parallel Gaussian sources. 
The parallel Gaussian sources model is a good first-order 
approximation of the output to a linear transform operating a 
subband decomposition of some natural source. For example, 
the JPEG2000 image coder [7] transforms the original image 
by using a discrete wavelet transform (DWT) and the first- 
order marginal statistics of the transform coefficients in the 
different subbands may be approximated by Gaussian random 
variables with different variances. 

Building on our previous works [3] and [8], we use multi- 
resolution (embedded) quantization and layered JSCC. The 
allocation of source layers to different users can be optimized 
according to different criteria, as discussed in Section [HI] 
Performing close to the theoretical limits requires coding rate 
adaptation with very fine granularity. This is accomplished by 
using "rateless" raptor encoders [9], which are able to produce 
any arbitrary number of coded symbols with a single, low 
complexity, encoding machine. The key idea of our practical 
JSCC scheme is that (see [10] and [11]) the rate-distortion (R- 
D) limit for a smooth source over a symmetric channel can 
be closely approached by using scalar quantization followed 
by a linear encoding function that maps directly the redundant 
quantization indices into channel input symbols. In the limit of 
arbitrarily large block length and unlimited coding/decoding 
complexity, this approach provably achieves the quantizer 
distortion Dq at bandwidth ratio H/C (channel uses per 
source sample) where C denotes the channel capacity and H 
denotes the entropy rate of the quantization indices, modeled 
as a discrete source. For a well-behaved source, the rate H 
achieved by entropy-coded quantization is close to the optimal 
rate TZ(Dq) (where IZ(-) denotes the R-D function of the 
source) [12], [13]. Hence, linear encoding of quantization 
indices performs close to optimal. The main advantage of the 



scheme, though, appears at finite block length and low encod- 
ing/decoding complexity. In fact, our approach eliminates the 
need for an explicit entropy coding stage after quantization. It 
is well-known that standard entropy coding is ill-conditioned 
with respect to residual channel decoding errors. In contrast, 
our scheme reconstructs directly the quantization symbols 
using a "soft-bit" approach, i.e., using the estimated posterior 
log-likelihood ratios (LLRs) generated by a Belief Propagation 
(BP) iterative decoder that incorporates the a priori statistics 
of the source (details can be found in [10], [11], [8] and 
[3]). In this way, the "catastrophic" behavior of entropy-coded 
quantization is greatly reduced. 

As noted above, we implement linear encoding by using 
raptor codes [9]. Recently, raptor codes have been standardized 
as application layer Forward Error Correction (FEC) coding 
for Multimedia Broadcast/Multicast Services (MBMS) within 
3GPP [14]. We hasten to say that our application is very 
different from this standard. In 3GPP [14], the "static broad- 
casting" [15] of a common content file is considered, where 
each user gathers channel observations until it has received 
non-erased symbols such that the whole file can be perfectly 
decoded. Users may have different decoding delays depending 
on their erasure probabilities. On the contrary, in this work we 
consider a truly "real-time" multicasting where the decoding 
delay is the same for all users, but each user reconstructs the 
source at a possibly different distortion level, depending on its 
own channel capacity. Our scheme can be naturally applied 
to static broadcasting (e.g., video on demand applications) 
by using it in conjunction with well-known protocols such 
as harmonic broadcasting [16], [17]. However, we do not 
investigate the details of this application in this paper. The 
numerical results of Section [V] show that the proposed coding 
scheme can achieve end-to-end distortion performance very 
close to the theoretical limits with finite block length and low 
encoding/decoding complexity. 

II. BEBC and Parallel Gaussian Sources 

The Binary Erasure Broadcast Channel (BEBC) has in- 
put alphabet {0,1}, output alphabet {0,1, e}, ("e" denoting 
erasure), and is defined by L channel transition probabilities 
P®{y\x) = l-eifory = x, P^(e\x) = e; and P®(y\x) =0 
for y ^ x, e. Without loss of generality, we let t\ > . . . > 
and denote by C; = 1 — e; the capacity of the l-th Binary 
Erasure Channel (BEC). This channel serves as a simple model 
for a multicast network, in which an arbitrarily large number 
of users are grouped into L classes, each characterized by a 
different channel capacity. For simplicity, we shall refer to 
each class as a "user" since, in a multicast scenario, all users 
belonging to the same class are indistinguishable and achieve 
the same performance. 

It is well-known that, under mild conditions on the erasure 
statistics the capacity region of the BEBC is given by [2] 



C = {(R 1 ,...,R L ):R l >0, £^<i, /e{l, 



i=i 
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This region is achieved by time-sharing between the vertices 
of the region's dominant face, defined by the hyperplane 
Y^,t—i §7 = 1- The BEBC belongs to the class of stochastically 
degraded broadcast channels [18]. This implies that any 
message to user I can be also decoded by all users j > I 
(i.e., by the users with better channels). 

The considered source model consists of s independent 
Gaussian source "components". A source block of length k 
is denoted by S e ]R sxfe , where its i th row, denoted by 
S( i: ), has i.i.d. elements SVjj) ~ A/"(0, erf), where we assume 
a\ > ■ ■ ■ > a\. Note that S contains s x k source symbols. 

The encoder maps S into a channel input codeword X = 
f {k) (S), where /<» : R sxk -> {0,1}" is the encoding 
function. At each Z-th user decoder, the received channel 
output Y; is mapped into a reconstructed source array Si = 
g ( i(Yi), where g[ k) : {0,1, e}" -> R sxk is a decoding 
function. The Mean Square Error (MSE) distortion for the I 



th 
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decoder and the i source component is D\ 
S(i. : ),z || 2 ]> where the expectation is with respect to the joint k- 
dimensional probability distribution of (S^.), S^w) induced 
by the source, by the channel erasures and by the coding 
scheme. The average total distortion at the I th decoder is given 

by opMEUaff- 

The R-D function of S is given by the "reverse waterfilling" 
parametric form [18]: 

1 8 1 o" 2 I s 

% ( D ) = I E 2 maX \ ' log2 ~}' D = 7s 2 min{7 ' 
s i=i " 7 s 

For each pair (R, D) there exist a pair (7, m) such that 
D = \{ im + E; =ro+1 o]Y and R = ± E™ 1 \ log 2 (a 2 / 7 ), 
with 7 < a 2 m . Similar to a single Gaussian source [19], 
it is easy to see that parallel Gaussian sources are also 
successively refinable under MSE distortion. In particular, 
letting (Ri,D\) and (i?2,^2) denote two points on the R- 
D function with D2 < D\, and letting (mi, 71) and (m.2,72) 
the corresponding parameters of the reverse waterfilling for- 
mula, the rate increment for each i-th source component is 
given by max{0, i log 2 (cr 2 /7 2 )} - max{0, i log 2 (cr 2 /7i)}, 
corresponding to a total rate increment per source symbol 
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where the first term corresponds to the extra quantization bits 
for the mi source components that are (coarsely) quantized to 
achieve distortion D\, and the second term corresponds to the 
additional quantization bits to quantize additional 7712 — mi 
source components that are discarded at distortion D\, but 
need to be encoded to achieve distortion D 2 . 

An ideal (R-D achieving) multiresolution source code pro- 
duces L layers for each source component at rates R^, for 
i = 1, . . . , s and / = 1, . . . ,L (some layers may have zero 
rate). The L layers are encoded by a broadcast code for the 
BEBC at rates R = (Ri,...,R L ) g C. Fig. [TJ shows the 
multiresolution layered coding scheme for the z th component. 



Inspired by the bit-per-pixel convention in image compres- 
sion literature, we define bandwidth expansion factor b as the 
ratio of the number of channel uses to the number of source 
symbols, i.e. b = n/(ks). For a binary-input channel such 
as the BEBC, b corresponds precisely to the ratio of (source- 
channel) coded bits per source symbol. Then for each layer 
I, we have = /3i=i R'i i = anc ' nence user I achieves the 
distortion 
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III. Optimization Problems 

A layered source-channel coding scheme can be optimized 
according to various criteria. Here we propose the following 
alternatives: 

Min Bandwidth (MB): For given target distortions d\ > 
■ ■ ■ > d,L, we wish to minimize b. Using (0) and (Q~|), the 
problem can be formulated as 



minimize 



subject to bS~\Rj > lZ(di), 



V/ 



3=1 
L 



Y-<i, 

i=i 1 



Ri > 0, V I 



(4) 



Letting TZ(do) = 0, we easily see that a necessary condition 
for optimality is that the inequalities bRi > H(di) — TZ(di-i) 
must hold with equality for all I = 1, . . . , L, Replacing these 
into the capacity region constraint, we obtain the solution 

L 



E 

i=i 



n{di)-n{di-- l ) 



(5) 



Min Weighted Total Distortion (MWTD): For given b and 
non-negative weights {wi}, we wish to minimize Yli=i w iDi. 

In [3], the case s = 1 is solved in closed form. The general 
case s > 1 treated here is more difficult. However, we can cast 
MWTD as a convex optimization problem with respect to the 
variables {Ri} and {Du} that can be solved by standard tools 
by incorporating the reverse waterfilling solution of the 1Z(D) 
into the problem itself. We have 

L 



minimize 



subject to 



1=1 \ i=l 



Ri 



L 
1=1 

< Du < a. 



< 1, Ri > 0, V/ 
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Min-Max Distortion Penalty (MMDP): For given 6, we wish 
to minimize max; e £ Di/D° pt , where D° pt = lZ~ 1 (bCi) is the 
individual R-D bound for user I. 



Fig. 2. D° pt for capacity values ranging from 0.1 to 0.9. The Di point 
(stars) represent the solutions of the MMDP problem for the capacities 
C = [0.3645, 0.81, 0.9] when 4 parallel sources with variances a 2 = 
[50, 12, 8, 5] are considered. 



Again, in [3] we treated the case s = 1 that can be cast as 
a simple linear program. In the general case s > 1, we obtain 
the following convex problem with respect to the variables a, 
{Ri} and {A,J : 



minimize 



subject to E 7^ - Rl - 



Ci 
i=i 

< Du < a? 



V/ 



vu 



i s -i 9 ' 

1=1 J=l 
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Rj, VI 
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In Fig. |2] we plot D° pt for capacity values ranging from 
0.1 to 0.9. The points D l represent the solution of the MMDP 
optimization problem for the capacities and the set of variances 
we use in Section IVl for the simulations. For each considered 
capacity, the distortion penalty of each user can observed by 
comparing the curve D° pt with the distortion points 

IV. Joint Source-Channel Coding for the BEBC 

In this section we present some code design examples that 
approach the theoretical limits obtained in ©, (|6]l and 
Generalizing the scheme proposed in [3], the layered JSCC 
scheme for parallel Gaussian sources is based on the concate- 
nation of scalar embedded quantizers with linear encoding 
functions, implemented by the same basic raptor encoding 
"machine". We use an embedded scalar quantizer Q : M — > 
{0, 1} , optimized for a Gaussian source ~ JV(0, 1). In order 
to quantize the z-th source component, the source symbols 
are scaled by l/crj and the inverse scaling of the quantizer 
reconstruction points is applied at the decoder. 

Let TJW = 0( S,. : t, i denote the sequence of quanti- 
zation indices, formatted as an TV x k binary array. The 
j-th row of U^, denoted by Uj., is referred to as the 
the j th "bit-plane". Without loss of generality, we let Uj } 
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Fig. 1. A multiresolution source encoder creates 3 layers with D^i > Di,2 > ^i,3 for three classes of users with capacities C\ < C*2 < C3. The channel 
encoder-decoder pair at rate C is shown as blocks CE(C) and CD(C), respectively. 
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Fig. 3. SNR vs. Rate, for a Gaussian source with a = 1, obtained by the R- 
D function (SNR r( ;(i?')), by the set of R-D points of the embedded quantizer 
with coordinates (ELi H i> D a(p)) for p = 1, . . . , 10 (SNR e9 (i?')) and 
by Dq(R') = Y2- 2R ' where V = 1.5, (SNR r (R')). 

denote the sign bit-plane, and XJ 2 U]y . denote the 
magnitude bit-planes with decreasing order of significance. 
The quantizer output IjW forms a discrete memoryless source, 
with entropy rate H"' = rif(UW) (in bits/source symbol). 
This can be decomposed as i?W — Y^jLi Hj > wner e the 
conditional entropy rates of the bit-planes are denoted by 
Hf = \H{X5f. |ug , . . . , uJ2 1>: ), for j = 1, ... , TV. The R- 
D function achieved by embedded scalar quantization followed 
by entropy coding of the quantization indices is given by the 
set of R-D points with coordinates (^J2^ = i DQj(p)j , 
for p = 1, ...,7V, where the quantizer distortion can be 
approximated by the following function Bg.j(r) w Taj 2 _2r , 
where T is a multiplicative penalty factor [12], [13] and for 
the i th source r = J2^=i Hj I s tne total rate of p bit-planes. 

We define reconstruction signal-to-noise-ratio SNR= 
101og 10 ^-.The SNR-R curve of the embedded scalar quan- 
tizer used in this work is shown in Fig.[3]for a = 1. We notice 
that the familiar 6dB per quantization bit improvement in the 
reconstruction SNR is achieved within a gap r« 1.5. 

The code design proceeds as follows: first, we approximate 
the solution of the desired optimization problem (e.g., MB, 
MWTR MMDP said above) by taking into account that the 
actual quantizer R-D function has gap T and that the source 
coding rates must be quantized into at most N discrete values, 



reflecting the bit-planes. The details of this approximation shall 
be given later on. For the time being, it is sufficient to say 
that the result of this first design step is a bit-plane allocation, 
defined by the integers {pi,i}, such that pu is the number of 
bit-planes of the z-th source component that must be used for 
source reconstruction at the Z-th decoder (we define p^o = 
for notation convenience). 

Then, for each I, we form the Z-th layer channel codeword by 
collecting all the quantization bit-planes with indices pn-\ < 
3 < Pi.i for a H source components i = 1, . . . , s, and mapping 
them independently onto binary codeword Xj j via a linear 
raptor encoder. Building on our results in [8], we use a 
systematic binary raptor encoder such that the source bit- 
planes form the systematic part, and the produced codeword 
corresponds to the "parity" symbols (see [9], [11], [8], [3] for 
details). We use the following degree distribution [9]: 

Q(x) = 0.008a; + 0.494x 2 + 0.166x 3 + 0.073a; 4 + 0.083a; 5 

+0.056a: 8 + 0.037a: 9 + 0.0562; 19 + 0.025a; 65 + 0.003a; 66 . 

Finally, the J2i=iPi,L codewords x^, . . . ,x SjPs L are trans- 
mitted in sequence. This corresponds to the time-sharing 
nature of the BEBC capacity region. We note here that the 
very same scheme can be applied with minor modifications to 
any underlying L-user broadcast channel, if one is willing to 
accept the performance penalty of using time-sharing instead 
of an optimal broadcast coding strategy. 

At each I th decoder, all codewords from (i, 1) to {i,Pij) for 
all source components i = 1 , . . . , s are sequentially decoded 
using the successive multi-stage decoder described in [8], [11]. 
Each (i,j)-th stage of the multistage decoder is an iterative 
Belief Propagation decoder that incorporates the information 
corresponding to the a priori probability distribution of the bit- 
plane symbols and the posterior log-likelihood ratios produced 
by the decoders at previous stages, (i, 1) . .., (i, j — 1). We 
omit the detailed description of the decoder because of space 
limitation. If the code is well designed, the distortion achieved 
at decoder I is essentially equal to the quantization distortion, 
given by A w E ^ 2~ 2E ^ H ? ■ 

In the rest of this section we discuss the details of the bit- 
plane allocation to the layers and of the choice of the coding 
rates. Let {D* ; } denote the set of distortions resulting from 
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MWTD, b = 8.75 
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the solution of one of the optimization problems of Section 
|m] We choose p M such that r<7?2 _25 ^=i w 

Since p^; must be an integer, this step requires some ad- 
hoc approximation. Recalling that the bit-planes with indices 
Pi,i-i < 3 < Pi.i must be decoded by user I, the block length 
of codeword x,j is given by 

where Si denotes a suitable "gap-to-capacity" (or "overhead") 
of the raptor code (see [9]). The overall bandwidth expansion 
factor of the scheme is given by 



1 
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V. Results 

In this section, we report a few simulation results for source 
block length k = 10000, with L — 3 users with capacities 
C_ = [0.3645, 0.81, 0.9] and s = 4 source components with 
variances a 2 = [50, 12, 8, 5]. 

For the MMDP optimization scenario, we fix b = 2.5. 
The optimal values of D* t obtained by solving are given 
in Tab. Q] (top). The allocation of bit-planes to layers are 
done as described earlier. The resulting Di j values and the 
bit-plane allocation variables pu' s are reported in Tab. Q] 
(bottom). As a sanity check, if we use the values Dij of 
Tab. U (bottom) to obtain target distortions di's in the MB 
optimization problem, we find that the required bandwidth 
expansion factor is b m i n = 2.58. Hence, the rounding of 
the optimal distortion values has only a marginal effect on 
the overall theoretical bandwidth efficiency. The actual linear 
coding and soft-bit reconstruction JSCC scheme, after careful 
optimization of the raptor code redundancy overhead, achieves 
the desired distortions at bjscc = 3.59. Similar results are 
reported for the MWTD optimization scenario, where we fix 
b = 8.75. The solution of © is reported in Tab. HH (top) and 
the actual code performance, obtained by simulation, is given 
in Tab. [TT] (bottom). 
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